A significant task, common in real-time image processing, is to detect objects that are moving with respect to stationary landscape. In particular, there is a need for a simplified method of detecting these moving objects when viewed by imaging sensors mounted on an aircraft in flight.
Historically, the preferred method of detecting moving objects is “frame subtraction.” The frame subtraction process takes two images, obtained with a time delay between them, registers them so that stationary features are mutually aligned, and subtracts one image from its companion. This difference will show changes, such as due to motion, as either positive or negative values. In many cases frame subtraction does not work well because motion of the camera induces significant parallax shifts which gives even stationary objects the appearance of “motion.” Further, the camera may change its orientation between the images, which also induces false motion of stationary objects.
A solution to the aforementioned problem has long been known. The technique is to select trackable stationary (i.e. terrain) features from the images and use these features to model the changes associated with all the stationary objects. In effect, the apparent movement of these stationary features provides information about the contours and perspective distortions of the observed terrain (including structures, such as buildings, trees, etc.). Once the terrain model has been established, it can be used to warp one of the images so that stationary features in the warped image overlay the corresponding stationary features in its companion image. Once this has been done, subtracting one frame from its companion reliably reveals objects which are moving with respect to the stationary terrain.
There are simplified versions of the foregoing modeling technique. One approach is to assume that the ground is essentially flat (true for most local observations). This ground plane approach provides the basic reference for the warping. In general, the ground plane approach stretches away from the camera so that a perspective distortion is inherent in the image of the ground plane. The benefit of this approach is that the ground plane seen by one camera is readily warped to match the same ground region as seen by the companion camera. The technique is well known in the literature as a “homography transformation.”
Structures which stick up from the ground plane are not warped the same way as the ground plane. These structures “sway” in such a way that their bases, which are on the ground plane, warp properly under homography. However, their elevated portions move in the imagery according to the rules of differential parallax. This apparent sway motion causes leakage during the frame subtraction which can defeat the detection of truly moving objects. One solution is to provide spatial filters which detect the spatial properties of these swaying objects and rejects them from the final frame subtraction.
The foregoing techniques of frame subtraction and warping as well as other conventional techniques for detecting true moving objects from aerial imagery, however, are computationally intensive.